System, apparatus and method for interactive reading

ABSTRACT

System, apparatus and method for facilitating interactive reading can include an electronic device having a program or application thereon. In one embodiment, the application can recognize one or more cues, combined with an external data source, that result from reading a story aloud and/or performing one or more acts.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional ApplicationNo. 62/361,644, filed Jul. 13, 2016 and entitled “System and Method forInteractive Reading,” which is herein incorporated by reference in itsentirety.

BACKGROUND

It is known to provide a device that responds to audio cues or readingaloud. For example, U.S. Publication No. 2006/0073444 discloses that thereader of a book use a coded indicia device to electronically scanindicia in a book to select an appropriate track on the device. Further,U.S. Publication No. 2009/0246749 discloses the use of a portableprojector and light signals that include ultraviolet light.

BRIEF SUMMARY

The presently disclosed technology overcomes drawbacks in the above andother prior art, and provides additional advantages. For example, in oneembodiment, the presently disclosed technology does not require that thebook or other reading material have any electronic identifiers thereonor therein. In one embodiment, the presently disclosed technologyemploys a touch screen display, which can display and interact withcontent that has been activated by spoken words from one or more users.

In one embodiment, the present disclosure relates generally to a systemand method that enables or facilitates interactive reading. In oneparticular embodiment, a computer application helps immerse a child intoa book or story. A reader's or viewer's imagination can be stretched orexpanded, thereby creating a unique experience and atmosphere, withoutdetracting from the reading of the book or story. As a story is readaloud, the application can bring to life the scenes and atmosphere ofthe book through sounds, colors, motion and the like, all of which canbe created from the application. These scenes (which can include soundeffects, still graphics, animations, etc.) can be triggered and alteredin response to external data sources obtained via the device, such asthe movement of the reader, geographic location, the time of day,automatic calendar of seasonal events, holidays and festivities, andother variable external data sources. This experience can encourage achild to continue to read and give children a reason to read. The readeris also able to create their own scenes for their own publications.

In another embodiment, the present disclosure relates to a system thatcan be configured to provide an interactive reading experience. Thesystem can include a microphone configured to convert sound into atleast one electrical signal, a speaker configured to convert at leastone electrical signal into sound, a display configured to convert atleast one electrical signal into one or more visual representations, oneor more processors operatively connected to the microphone, the speakerand the display, and one or more memories operatively coupled to the oneor more processors and having computer readable instructions storedthereon which, when executed by at least one of the one or moreprocessors, causes the one or more processors to receive a plurality ofdistinct audible cues through the microphone from an individual readingaloud from a predetermined script, and produce a plurality of responsesthrough at least one of the speaker and display, each response of theplurality of responses being based on one of the plurality of audiblecues.

In yet another embodiment, the present disclosures relates toanon-transitory computer-readable medium having computer-readable codestored thereon that, when executed by one or more computing devices,causes the one or more computing devices to receive, by at least one ofthe one or more computing devices, a first audible cue from anindividual reading aloud from a predetermined script; produce, by atleast one of the one or more computing devices, a first response throughan electronic device based on the first audible cue; receive, by atleast one of the one or more computing devices, a second audible cuefrom the individual reading aloud from a predetermined script, thesecond audible cue being different than the first audible cue; andproduce, by at least one of the one or more computing devices, a secondresponse through the electronic device based on the second audible cue,the second response being different than the first response.

In still a further embodiment, the present disclosures relates to methodimplemented by one or more computing devices for creating an interactivereading experience. The method can include receiving, by at least one ofthe one or more computing devices, one or more cues from an individualreading aloud from a predetermined script; and producing, by at leastone of the one or more computing devices, a response based on each oneof the one or more cues. A response associated with a first one of theone or more cues can be different than a response associated with asecond one of the one or more cues.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofthe invention, will be better understood when read in conjunction withthe appended drawings. For the purpose of illustrating the invention,there are shown in the drawings various illustrative embodiments. Itshould be understood, however, that the invention is not limited to theprecise arrangements and instrumentalities shown. In the drawings:

FIG. 1 shows a multitude of components and steps according to anembodiment of the present disclosure;

FIG. 2 shows a multitude of components and steps according to anembodiment of the present disclosure;

FIG. 3 shows a multitude of components and steps according to anembodiment of the present disclosure;

FIG. 4 shows several steps according to an embodiment of the presentdisclosure;

FIG. 5 shows a multitude of components and steps according to anembodiment of the present disclosure;

FIG. 6 shows an exemplary computing device useful for performingprocesses disclosed herein; and

FIG. 7 shows several steps according to an embodiment of the presentdisclosure.

DETAILED DESCRIPTION

Certain terminology is used in the following description for convenienceonly and is not limiting. Certain words used herein designate directionsin the drawings to which reference is made. Unless specifically setforth herein, the terms “a,” “an” and “the” are not limited to oneelement, but instead should be read as meaning “at least one.” Theterminology includes the words noted above, derivatives thereof andwords of similar import.

Referring to the drawings in detail, wherein like reference numeralsindicate like elements throughout, FIGS. 1-7 illustrate a system andmethod designed to facilitate or encourage interactive reading. Thesystem and method can include an electronic device, such as a hand-heldmobile phone or a tablet or other computing device, having a program orapplication thereon or accessible thereto. The application and/or one ormore portions of the electronic device can recognize one or more audiblecues that result from reading a story aloud together with data-point(s)to then generate responses thereto. Stated differently, the applicationand/or one or more portions of the electronic device can employ voicerecognition technology (e.g., a voice recognition chip, a microphone, aspeaker, and a general purpose or application specific processor orintegrated circuit) that can “listen” for audible cues, which, whenspoken, trigger the application to automatically play or launch specificcontent through the electronic device. The specific content is designedto be relevant to the story and environment.

In operation of one embodiment, as a user or reader reads the storyaloud, the application can identify and/or recognize one or more audiblecues (e.g., sounds, words, phrases, sentences, inflections, etc.) andone or more sensed or received data-points or other externalinformation, and produces one or more responses to same. The one or moreresponses can be to display an image, animation, video and/or game on ascreen of the electronic device, and/or produce an audible noise or songthrough one or more speakers of the computer device, vibrate, flashand/or the like. The one or more responses can complement or enhance thestory, and add or increase reader/viewer engagement. The one or moreresponses can be affected by the external data-source to be morepersonalized to the user's environment.

The story may be read from a book, a magazine, a publication, anewspaper, an electronic book, a website, a computer application, a game(e.g., a board game), a billboard or the like. The electronic device isnot limited to being a hand-held mobile phone or a tablet. Instead, theelectronic device may be a laptop computer, a desktop computer, atelevision, a wearable electronic device (e.g., a smart watch) or thelike. While the application and/or one or more portions of theelectronic device can recognize audible cues, alternatively oradditionally the application and/or one or more portions of theelectronic device can recognize visual cues (e.g., through a camera ofthe electronic device) and/or physical cues (e.g., shaking or twistingof the electronic device). The visual and/or physical cues may trigger aresponse from the application and/or electronic device.

FIGS. 1 and 2 show embodiments of the present disclosure. For example,when the phrase “bottles and jars of every sort” is read aloud, theapplication and/or one or more portions of the electronic device canidentify and/or sense that, and can cause the speaker of the electronicdevice to emit a sound of jars rattling and clanking. Alternatively oradditionally, the application and/or one or more portions of theelectronic device can cause the speaker to emit glugging and rattlingnoises after the application senses that ingredients are read aloud. Asingredients are read aloud, the application and/or one or more portionsof the electronic device can cause the display of the computerelectronic to appear to fill with liquid and display steam, show apicture of a crocodile and/or display spinning imagery. Of course, anyof a plurality of responses (examples of which are described above) canbe created or performed. In one embodiment, each response can last foronly a few seconds, such that a plurality of responses can be played inseries as the story is read.

In one embodiment, multiple sets of triggers or responses could becreated for each story.

Therefore, each time a story is read, different elements of the storymay be addressed or brought to life by the application. For example, thefirst time a particular story is read, the application and/or one ormore portions of the electronic device can generate three out of tenresponses. When the same story is read for a second time, theapplication and/or one or more portions of the electronic device canremember (e.g., through memory) which three responses have already beengenerated and can generate additional or alternative responses.

In one embodiment, when the same story is read using a differentdata-point, e.g., a different geo-location, a different time of day, adifferent season, a different individual or reader, a different speed atwhich the story is read aloud, a different voice inflection by thereader, etc., the application can produce one or more variants of theresponse. For example, one or more scenes and/or sounds can be triggeredand altered by external data sources obtained via the device, such asthe movement of the reader, the reader's voice inflection, geographiclocation, the time of day, automatic calendar of seasonal events,holidays and festivities, and other variable external data sources.

One example of this embodiment would be a user reading about anadventure where a character is attempting escape on a horse. The devicecould respond with the sound of a horse galloping, but the pace and/orvolume of the gallop sound could be affected by the movement of thedevice, i.e., fast device movement or fast shaking of the cell-phoneresults in a faster gallop.

Another example of this embodiment including the playing of festivemusic in the background during the month of December via the devicebeing fed data of the current date.

FIG. 3 shows another embodiment of the present disclosure, which directsthe user or reader to complete or view certain activities on theelectronic device. For example, when the phrase “now it's your turn” isread aloud, the application and/or one or more portions of theelectronic device can cause the display of the electronic device to showa series of levers and buttons that may be manipulated or moved by auser touching specific portions of the display. Upon sensing the usertouching one or more of the buttons, the application may cause thedisplay to play an animated video and/or the speaker to emit soundsand/or songs. The video may be different depending upon which buttons orlevers are selected by the user or reader.

FIG. 4 shows an embodiment of a method, generally designated 400, of thepresent disclosure. The user or reader can purchase or obtain a product,such as a book and/or a Happy Meal® (Step 402). The product can includeinformation or directions (e.g., on an instruction card, book or toy)for the user to install or download (e.g., from the Internet) anapplication onto an electronic device (Step 404). The user may or maynot need to register or log-in to the application

(Step 406). The user can begin the interactive reading exercise byselecting or clicking one or more icons on the display of the electronicdevice (Step 408). Alternatively, the application can automaticallyload. In either case, the application can be prepared to listen or sensecues from the user or reader.

FIG. 5 shows an embodiment of method of the present disclosure once theapplication has been loaded or activated. Initially, the applicationand/or one or more portions of the electronic device can cause thedisplay of the electronic device to prompt a user or reader to select astory (Step 502). In one embodiment, the list of stories available ordisplayed can be restricted based on a predetermined time period (e.g.,each week). For examples, two stories may be available during the firstweek, and two additional stories may be available each week thereafter.For each week, the application can load story specific assets orinformation.

Once a story is selected, the application and/or one or more portions ofthe electronic device can cause the display of the electronic device toprovide instructions or tips to the user or reader (Step 504). Anexample of a tip is to read each word of the story out load, slowly andclearly for best results. The user can then select a “Begin” or “Start”button to cause the application to begin “listening” for the audiblecues. Of course, users or readers can skip ahead or go back usingappropriate controls on the display. The interactive reading method cancontinue as described above and shown in FIGS. 1-3.

One or more of the above-described systems and/or methods may beimplemented with or involve software, for example modules executed onone or more electronic devices 610 (see FIG. 6). Of course, modulesdescribed herein illustrate various functionalities and do not limit thestructure or functionality of any embodiments. Rather, the functionalityof various modules may be divided differently and performed by more orfewer modules according to various design considerations.

Each electronic device 610 can include one or more processing devices611 designed to process instructions, for example computer readableinstructions (i.e., code), stored in a non-transient manner on one ormore storage devices 613. By processing instructions, the processingdevice(s) 611 may perform one or more of the steps and/or functionsdisclosed herein. Each processing device may be real or virtual. In amulti-processing system, multiple processing units may executecomputer-executable instructions to increase processing power. Thestorage device(s) 613 may be any type of non-transitory storage device(e.g., an optical storage device, a magnetic storage device, a solidstate storage device, etc. The storage device(s) 613 may be removable ornon-removable, and includes magnetic disks, magnetic tapes or cassettes,CD-ROMs, CD-RWs, DVDs, or any other medium which can be used to storeinformation. Alternatively, instructions may be stored in one or moreremote storage devices, for example storage devices accessed over anetwork or the interne.

Each electronic device 610 additionally can have memory 612, one or moreinput controllers 616, one or more output controllers 615, and/or one ormore communication connections 640. The memory 612 may be volatilememory (e.g., registers, cache, RAM, etc.), non-volatile memory (e.g.,ROM, EEPROM, flash memory, etc.), or some combination thereof. In atleast one embodiment, the memory 612 may store software implementingdescribed techniques.

An interconnection mechanism 614, such as a bus, controller or network,may operatively couple components of the computing device 610, includingthe processor(s) 611, the memory 612, the storage device(s) 613, theinput controller(s) 616, the output controller(s) 615, the communicationconnection(s) 640, and any other devices (e.g., network controllers,sound controllers, etc.). The output controller(s) 615 may beoperatively coupled (e.g., via a wired or wireless connection) to one ormore output devices 620 (e.g., a monitor, a television, a mobile devicescreen, a touch-display, a printer, a speaker, etc.) in such a fashionthat the output controller(s) 615 can transform the display on thedevice (e.g., in response to modules executed).

The input controller(s) 616 may be operatively coupled (e.g., via awired or wireless connection) to an input device 630 (e.g., a mouse, akeyboard, a touch-pad, a scroll-ball, a touch-display, a pen, a gamecontroller, a voice input device, a scanning device, a digital camera,etc.) in such a fashion that input can be received from a user.

The communication connection(s) 640 enable communication over acommunication medium to another computing entity. The communicationmedium conveys information such as computer-executable instructions,audio or video information, or other data in a modulated data signal. Amodulated data signal is a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia include wired or wireless techniques implemented with anelectrical, optical, RF, infrared, acoustic, or other carrier.

FIG. 6 illustrates the electronic device 610, the output device 620, andthe input device 630 as separate devices for ease of identificationonly. However, the electronic device 610, the display device(s) 620,and/or the input device(s) 630 may be separate devices (e.g., a personalcomputer connected by wires to a monitor and mouse), may be integratedin a single device (e.g., a mobile device with a touch-display, such asa smartphone or a tablet), or any combination of devices (e.g., acomputing device operatively coupled to a touch-screen display device, aplurality of computing devices attached to a single display device andinput device, etc.). The electronic device 610 may be one or moreservers, for example a farm of networked servers, a clustered serverenvironment, or cloud service(s) running on remote computing devices. Adata-source 650 can provide data, such as from one or more externalsources, to the device 610. The data-source 650 can be external to thedevice 610 and can be provided by or coupled to a communication method,such as the Internet.

FIG. 7 shows an embodiment of a method, generally designated 700, of thepresent disclosure. A data source or multiple data sources (n) can beinputted into the device (Step 702). Alternatively, the device canreceive data from one or more data sources. The device can continuouslyreceive and/or listen to events or information from the data source(s)(Step 704). The device can compare the events or information to thosestored on the device (Step 706). If there is a match between the storedevent/information and the received data event/information, the devicecan then process or initiate a saved response against the current stateof the data source to produce a new and unique response (Step 708 andStep 710). The device can then output a final response, which may bepresented visually, audibly and/or physically (Step 712).

In contrast to prior art systems and methods, the system and method ofthe present disclosure does not require users to hold the phone in acertain position or orientation to activate augmented reality and/ormarkers within a book. The system and method of the present disclosuredoes not rely on pre-printed targets. The system and method of thepresent disclosure is also not limited to relying on a camera of theelectronic device to sense cues, as the present disclosure may employaudio or motion sensors in addition to or in place of reliance solely onthe camera, to sense cues from the user. The system and method of thepresent disclosure can effectively extend a user's interest in a story,because the application can produce different reactions to the samecues.

It will be appreciated by those skilled in the art that changes could bemade to the embodiments described above without departing from the broadinventive concept thereof. It is understood, therefore, that thisdisclosure is not limited to the particular embodiments disclosed, butit is intended to cover modifications within the spirit and scope of thepresent disclosure as defined by the appended claims.

1. A system configured to provide an interactive reading experience, the system comprising: a microphone configured to convert sound into at least one electrical signal; a speaker configured to convert at least one electrical signal into sound; a display configured to convert at least one electrical signal into one or more visual representations; one or more processors operatively connected to the microphone, the speaker and the display; and one or more memories operatively coupled to the one or more processors and having computer readable instructions stored thereon which, when executed by at least one of the one or more processors, causes the one or more processors to: receive a plurality of distinct audible cues through the microphone, the plurality of distinct audible cues emanating from an individual reading aloud from a predetermined script; and produce a plurality of responses through at least one of the speaker and display, each response of the plurality of responses being based on one of the plurality of audible cues.
 2. The system of claim 1, wherein the plurality of responses are serially manifested through at least one of the speaker and display.
 3. The system of claim 1, wherein, when executed by at least one of the one or more processors, the one or more memories causes the one or more processors to: receive data other than the plurality of audible cues, the data including information generated from at least one of a Global Positioning System (GPS) locator, a motion sensor, a clock and a calendar; and produce one or more responses based on the data, wherein at least one of the one or more responses based on the data is distinct from each of the plurality of responses based on the plurality of audible cues.
 4. The system of claim 1, wherein the plurality of audible cues includes at least one sound, word, phrase, sentence and voice inflection of the individual reading aloud.
 5. The system of claim 1, wherein the plurality of responses includes at least one of (i) an image on the display, (ii) an animation on the display, (iii) a sound emanating from the speaker, (iv) vibration, (v) shaking, and (vi) flashing lights.
 6. The system of claim 1, wherein the predetermined script is one of a book, a magazine, a publication and a website.
 7. The system of claim 1, further comprising: a camera configured to record or capture one or more images or videos, the camera being operatively connected to the one or more processors, wherein, when executed by at least one of the one or more processors, the one or more memories causes the one or more processors to: receive one or more visual cues from an external environment around the electronic device; and produce one or more responses through at least one of the speaker and display, each of the one or more responses being based on the one or more visual cues.
 8. The system of claim 1, wherein consecutive, complete readings of the predetermined script result in different responses.
 9. A non-transitory computer-readable medium having computer-readable code stored thereon that, when executed by one or more computing devices, causes the one or more computing devices to: receive, by at least one of the one or more computing devices, a first audible cue from an individual reading aloud from a predetermined script; produce, by at least one of the one or more computing devices, a first response through an electronic device based on the first audible cue; receive, by at least one of the one or more computing devices, a second audible cue from the individual reading aloud from a predetermined script, the second audible cue being different than the first audible cue, the second audible cue received after receipt of the first audible cue; and produce, by at least one of the one or more computing devices, a second response through the electronic device based on the second audible cue, the second response being different than the first response.
 10. The medium of claim 9, wherein the first response includes one or more first images or animations being shown on a display of the electronic device, and wherein the second response includes one or more second images or animations being shown on the display of the electronic device.
 11. The medium of claim 9, wherein the first response is one or more first sounds being played from a speaker of the electronic device, and wherein the second response includes one or more second sounds being played from the speaker of the electronic device.
 12. The medium of claim 9, wherein, when executed by the one or more computing devices, causes the one or more computing devices to: receive data other than the one or more audible cues, the data including information from at least one of a Global Positioning System (GPS) locator, a motion sensor, and a clock; and produce one or more third responses based on the data.
 13. The medium of claim 12, wherein the one or more data sources include at least one Global Positioning System (GPS) locator, a motion sensor, and a clock.
 14. The medium of claim 9, wherein the one or more audible cues include at least one of sounds, words, phrases, sentences and voice inflections.
 15. A method implemented by one or more computing devices for creating an interactive reading experience, the method comprising: a) receiving, by at least one of the one or more computing devices, one or more cues from an individual reading aloud from a predetermined script; and b) producing, by at least one of the one or more computing devices, a response based on each one of the one or more cues, wherein a response associated with a first one of the one or more cues is different than a response associated with a second one of the one or more cues.
 16. The method of claim 15, wherein at least one of the responses is an audible response manifested through a speaker.
 17. The method of claim 15, wherein at least one of the responses is a visual response manifested through a display.
 18. The method of claim 15, further comprising: downloading, by at least one of the one or more computing devices, an application to an electronic device prior to step a); and initiating, by at least one of the one or more computing devices, the application on the electronic device prior to step a).
 19. The method of claim 15, further comprising: producing, by at least one of the one or more computing devices, a physical response based on each one of the one or more cues.
 20. The method of claim 19, wherein the physical response includes at least one of vibration and shaking of the electronic device, and wherein the visual response includes at least one of flashing lights, an image on a display, and a video on the display.
 21. The method of claim 15, further comprising: receiving data other than the plurality of audible cues, the data including information generated from at least one of a Global Positioning System (GPS), a motion sensor, a clock and a calendar.
 22. The method of claim 21, further comprising: comparing the received data to stored information; and in the event of a match between the data and the stored information, producing one or more responses based on the data. 